Finding Local Models in Classification (Extended Abstract)

It is commonplace knowledge that more and more data is collected everywhere and that the size of data sets available for knowledge discovery is increasing exponentially. On the one hand this is good, because learning with high-dimensional data and complex dependencies may need a large number of examples to give accurate results. On the other hand, there are several learning problems which cannot be thoroughly solved by applying a standard learning algorithm to all available examples. While the accuracy of the learner typically increases with example size, there may be many other criteria for a learning algorithm like speed, robustness or comprehensibility which are negatively affected by too much examples. What can we do about this problem? Often one can efficiently find a simple model which provides not an optimal solution, but a reasonably good approximation. The hard work usually lies in improving an already good model. Hence, we can try to find a simple model first and then concentrate on improving only those parts of the input space, where the model is not good enough. This will be an easier task because less examples have to be considered and hence one might use a more sophisticated learner. In other words, one constructs not one single global model for all the data, but a global model plus one or more local models to cover special cases. There is an obvious connection between local models and the detection of local patterns, which is defined as the unsupervised detection of high-density regions in the data [Hand, 2002]: given a global model the goal is to identify structures, i. e. to find high-density regions, in the errors of this model. The difference is that here the goal is not just to identify these regions, but to use them to improve the overall performance, e. g. the classification accuracy. Besides the obvious improvements in efficiency, the local model approach has an interesting implication for the aspect of interpretability and interestingness: as humans are very limited in the level of complexity they can intuitively understand [Miller, 1956], a simple model may give the user an useful intuition without getting into too much detail. On the other hand, for the aspect of discovering new knowledge, it may happen that the global model finds only the obvious pat-